Method and apparatus for encoding/decoding an image

ABSTRACT

Disclosed are an image encoding/decoding method and apparatus. The method and apparatus select at least one reference pixel line from among multiple reference pixel lines and derive a predicted value of a pixel within a current block by using the value of at least one pixel within the selected reference pixel line(s). Alternatively, the method and apparatus derive an intra prediction mode of a reconstructed pixel region on the basis of a reference pixel region of at least one reconstructed pixel region, derive an intra prediction mode of a current block on the basis of the derived intra prediction mode of the reconstructed pixel region, obtain an intra prediction block of the current block by using the derived intra prediction mode, and reconstruct the current block by summing the obtained intra prediction block and a residual block of the current block.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation application of U.S. patentapplication Ser. No. 17/318,020, filed on May 12, 2021, which is aContinuation application of U.S. patent application Ser. No. 16/341,590,filed on Apr. 12, 2019, now U.S. Pat. No. 11,039,148, which is a U.S.National Stage Application of International Application No.PCT/KR2017/011219, filed on Oct. 12, 2017, which claims the benefitunder 35 USC 119(a) and 365(b) of Korean Patent Application No.10-2016-0133753, filed on Oct. 14, 2016, Korean Patent Application No.10-2016-0133755, filed on Oct. 14, 2016, Korean Patent Application No.10-2017-0127938, filed on Sep. 29, 2017 and Korean Patent ApplicationNo. 10-2017-0127940, filed on Sep. 29, 2017 in the Korean IntellectualProperty Office, the entire disclosures of which are incorporated hereinby reference for all purposes.

TECHNICAL FIELD

The present invention relates to a video signal encoding/decoding methodand apparatus. More particularly, the present invention relates to animage encoding/decoding method and apparatus using improved intraprediction.

BACKGROUND ART

In recent years, demand for multimedia data such as video has beenrapidly increasing on the Internet. However, it is hard for developmentof technology for improving channel bandwidths to keep up with the rapidchanges in the demand for multimedia data. In order to solve thisproblem, VCEG (Video Coding Expert Group) of ITU-T which is the internalorganization for standard and MPEG (Moving Picture Expert Group) ofISO/IEC are continuously collaborating to establish improved videocompression standards.

Video compression consists largely of intra prediction, interprediction, transform, quantization, entropy coding, and in-loopfiltering. Among them, intra prediction is a technique of generating aprediction block for a current block by using reconstructed pixelsexisting around the current block.

Conventional intra prediction generates pixels at sub-pixel positionsthrough an interpolation process on the basis of reference pixels atinteger-pixel positions, and generates prediction blocks using thepixels at the generated sub-pixel positions. In this case, depending onwhich integer-position reference pixel is used and which interpolationscheme is applied, the error between the original pixel value and thepredicted pixel value is affected.

In addition, the conventional intra prediction technique needs to encodea significant amount of information about prediction modes in order toinform a video decoding apparatus that which intra prediction mode amongmultiple intra prediction modes is used for intra prediction of an inputimage.

DISCLOSURE Technical Problem

An object of the present invention is to improve intra predictionefficiency in an image encoding or decoding process by performing intraprediction with the use of multiple reference pixel lines, in encodingor decoding an image.

Another object of the present invention is to improve intra predictionefficiency in an image encoding or decoding process by generating anintra prediction block with the use of an interpolation schemeadaptively selected from among multiple interpolation schemes.

A further object of the present invention is to provide a filteringmethod capable of reducing discontinuity between an intra predictionblock and a surrounding region when intra prediction is performed usingmultiple reference pixel lines, in an image encoding or decodingprocess.

A yet further object of the present invention is to improve intrapicture prediction efficiency in an image encoding or decoding processby deriving an intra prediction mode used to encode or decode an imageby using an already reconstructed pixel region.

Technical Solution

According to an embodiment of the present invention, an image decodingmethod and apparatus may select at least one reference pixel line fromamong multiple reference pixel lines and derives a predicted value of apixel within a current block by using a value of at least one pixelwithin the selected reference pixel line(s).

According to an embodiment of the present invention, the image decodingmethod and apparatus may obtain reference pixel line index informationfrom an input bitstream and select the at least one reference pixel linefrom the multiple reference pixel lines on the basis of the referencepixel line index information.

According to an embodiment of the present invention, the image decodingmethod and apparatus may select at least one reference pixel line foreach pixel within the current block on the basis of a position of acorresponding one of the pixels within the current block.

According to an embodiment of the present invention, the image decodingmethod and apparatus may select an interpolation scheme from amongmultiple interpolation schemes and perform interpolation with at leastone pixel included in the at least one reference pixel line that isselected by using the selected interpolation scheme, thereby obtainingthe predicted value. The selected interpolation scheme may be selectedon the basis of index information indicating one of the multipleinterpolation schemes.

According to an embodiment of the present invention, the image decodingmethod and apparatus may obtain a prediction block by deriving thepredicted values of all the pixels within the current block and filterthe prediction block.

According to an embodiment of the present invention, the image decodingmethod and apparatus may filter a predetermined region of the currentblock, depending on a size of the current block or an intra predictionmode of the current block.

According to an embodiment of the present invention, an image encodingmethod and apparatus may select at least one reference pixel line fromamong multiple reference pixel lines and derive a predicted value of apixel within a current block by using a value of at least one pixelwithin the selected reference pixel line(s).

According to an embodiment of the present invention, the image encodingmethod and apparatus may encode reference pixel line index informationindicating the at least one pixel line that is selected and insert theencoded reference pixel line index information into a bitstream.

According to an embodiment of the present invention, the image encodingmethod and apparatus may select at least one reference pixel line foreach pixel within the current block on the basis of a position of acorresponding one of the pixels within the current block.

According to an embodiment of the present invention, the image encodingmethod and apparatus may select at least one reference pixel line foreach pixel within the current block on the basis of an intra predictionmode of the current block.

According to an embodiment of the present invention, the image encodingmethod and apparatus may select an interpolation scheme from amongmultiple interpolation schemes and perform interpolation with at leastone pixel included in the at least one reference pixel line that isselected by using the selected interpolation scheme, thereby obtainingthe predicted value.

According to an embodiment of the present invention, the image encodingmethod and apparatus may encode index information indicating one of themultiple interpolation schemes and insert the encoded index informationinto a bitstream.

According to an embodiment of the present invention, the image encodingmethod and apparatus may obtain a prediction block by deriving thepredicted values of all the pixels within the current block and filterthe prediction block.

According to an embodiment of the present invention, the image encodingmethod and apparatus may filter a predetermined region of the currentblock, depending on a size of the current block or an intra predictionmode of the current block.

According to an embodiment of the present invention, an imageencoding/decoding method and apparatus derives an intra prediction modeof a reconstructed pixel region on the basis of a reference pixel regionof at least one reconstructed pixel region, derives an intra predictionmode of a current block on the basis of the derived intra predictionmode of the reconstructed pixel region, obtains an intra predictionblock of the current block by using the derived intra prediction mode,and reconstructs the current block by summing the obtained intraprediction block and a residual block of the current block.

According to an embodiment of the present invention, the image decodingmethod and apparatus may obtain information indicating an intraprediction mode derivation method from an input bitstream and determinewhether to derive an intra prediction mode of a reconstructed pixelregion according to the obtained information indicating the intraprediction mode derivation method.

According to an embodiment of the present invention, the image decodingmethod and apparatus may obtain available intra prediction modeinformation indicating the number of multiple available intra predictionmodes or a list of the multiple available intra prediction modes andderive the intra prediction mode of the current block on the basis ofthe available intra prediction mode information.

According to an embodiment of the present invention, the image encodingmethod and apparatus may encode information indicating an intraprediction mode derivation method for a current block and insert theencoded information into a bitstream, and the image decoding method andapparatus may selectively perform derivation of an intra prediction modeof a reconstructed pixel region according to the information indicatingthe intra prediction mode derivation method for the current block.

Advantageous Effects

According to the present invention, it is possible to improve thecompression efficiency of an image and the image quality of a reproducedimage by using a more effective intra prediction technique. According tothe present invention, it is possible to improve the image quality of areproduced image by using a filtering method capable of reducingdiscontinuity between an intra prediction block and the surroundingarea.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an image encoding apparatusaccording to one embodiment of the present invention;

FIG. 2 is a diagram illustrating an example of an intra prediction mode;

FIG. 3 is a diagram illustrating a planar mode;

FIG. 4 is a diagram illustrating a DC mode;

FIG. 5 is a diagram illustrating an example of a method of generating aprediction block;

FIG. 6 is a block diagram illustrating an image decoding apparatusaccording to one embodiment of the present invention;

FIGS. 7A and 7B are diagrams illustrating a method of generating anintra prediction pixel by using an interpolation scheme;

FIG. 8 is a diagram illustrating an implicit method for selecting aninterpolation scheme or interpolation coefficients;

FIG. 9 is a flowchart illustrating a process in which an image encodingapparatus selects a mode for intra prediction;

FIG. 10 is a flowchart illustrating a process in which an image encodingapparatus selects an interpolation scheme from among multipleinterpolation schemes;

FIG. 11 is a flowchart illustrating a process in which an image encodingapparatus encodes information of an index indicating an interpolationscheme;

FIG. 12 is a flowchart illustrating a process in which an image decodingapparatus decodes information of an index indicating an interpolationscheme;

FIG. 13 is a diagram illustrating a process of deriving an intraprediction pixel by using multiple reference pixel lines, according toan embodiment of the present invention;

FIG. 14 is a flowchart illustrating a process of deriving an intraprediction pixel value, according to an embodiment of the presentinvention;

FIG. 15 is a flowchart illustrating a process of adaptively determininga reference pixel line to be used for intra prediction for eachprediction block;

FIG. 16 is a flowchart illustrating a process in which reference pixelline index information is encoded by an image encoding apparatus;

FIG. 17 is a flowchart illustrating a process in which the referencepixel line index information is decoded by an image decoding apparatus;

FIGS. 18 and 19 are diagrams illustrating a method of determining areference pixel line without transmitting a reference pixel line index;

FIG. 20 is a diagram illustrating smoothing between a prediction blockand a reference pixel line;

FIG. 21 shows a case where a reference pixel line is used for alltransform blocks within a current block;

FIGS. 22A to 22D show a case where a reference pixel line is selectedper transform block and the selected reference pixel line is used forintra prediction;

FIG. 23 is a diagram illustrating DIMD according to a first embodimentof the present invention;

FIG. 24 is a diagram illustrating DIMD according to a third embodimentof the present invention;

FIG. 25 is a flowchart illustrating DIMD according to the presentinvention;

FIG. 26 is a flowchart illustrating a method of encoding an intraprediction mode when an image is encoded using the DIMD according to thepresent invention;

FIG. 27 is a flowchart illustrating a method of decoding an intraprediction mode when an image is decoded using the DIMD according to thepresent invention;

FIG. 28 is a diagram illustrating a seventh embodiment of the presentinvention;

FIG. 29 is a flowchart illustrating a process of encoding an intraprediction mode when the seventh embodiment is used;

FIG. 30 is a flowchart illustrating a process of decoding an intraprediction mode when the seventh embodiment is used;

FIGS. 31A and 31B are diagrams illustrating modifications to a DIMD, inwhich a template index is transmitted;

FIG. 32 is a flowchart illustrating a method of coding an intraprediction mode according to a DIMD in which a template index is used;

FIG. 33 is a flowchart illustrating a method of decoding an intraprediction mode according to a DIMD in which a template index is used;

FIG. 34 is a diagram illustrating an example of a method of setting anintra prediction mode derived by using a template as an MPM candidate;and

FIG. 35 is a diagram illustrating a method of setting an MPM candidateaccording to an embodiment of the present invention.

BEST MODE

The present invention may be embodied in many forms and have variousembodiments. Thus, specific embodiments will be illustrated in theaccompanying drawings and described in detail below. While specificembodiments of the invention will be described herein below, they areonly illustrative purposes and should not be construed as limiting tothe present invention. Accordingly, the present invention should beconstrued to cover not only the specific embodiments but also cover allmodifications, equivalents, and substitutions that fall within the spritand technical scope of the present invention. Throughout the drawings,like elements are denoted by like reference numerals.

Terms used in the specification, “first”, “second”, etc., may be used todescribe various components, but the components are not to be construedas being limited to the terms. That is, the terms are used todistinguish one component from another component. For example, a firstconstitutive element may be referred as a second constitutive element,and the second constitutive element may be also referred to as the firstconstitutive element. Moreover, the term “and/or” includes any and allcombinations of one or more of the associated listed items.

It will be understood that when any element is referred to as being“connected” or “connected” to another element, one element may bedirectly connected or coupled to the other element, or an interveningelement may be present therebetween. In contrast, it should beunderstood that when an element is referred to as being “directlycoupled” or “directly connected” to another element, there are nointervening elements present.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used herein, thesingular forms “a”, “an” and “the” are intended to include the pluralforms as well unless the context clearly indicates otherwise. It will befurther understood that the terms “comprises”, “includes”, or “has” whenused in this specification specify the presence of stated features,regions, integers, steps, operations, elements and/or components, but donot preclude the presence or addition of one or more other features,regions, integers, steps, operations, elements, components and/orcombinations thereof.

Hereinafter, exemplary embodiments of the present invention will bedescribed in detail with reference to the accompanying drawings.Hereinafter, like constituent elements are denoted by like referencenumerals throughout the drawings, and redundant explanations for thesame constituent elements will be omitted.

FIG. 1 is a block diagram illustrating an image encoding apparatusaccording to one embodiment of the present invention.

Referring to FIG. 1 , an image encoding apparatus 100 includes an imagepartition unit 101, an intra prediction unit 102, an inter predictionunit 103, a subtractor 104, a transformation unit 105, a quantizationunit 106, an entropy encoding unit 107, a dequantization unit 108, aninverse transformation unit 109, an adder 110, a filter unit 111, and amemory unit 112.

In addition, components described in exemplary embodiments of thepresent invention are independently shown only in order to indicate thatthey perform different characteristic functions in an image encodingapparatus. Therefore, the components that are independently shown do notmean that each of the components is implemented as one piece of hardwareor software. That is, each of the components is divided for convenienceof explanation, multiple components may be combined with each other tothereby be operated as one component or one component may be dividedinto a plurality components to thereby be operated as the multiplecomponents, which are included in the scope of the present invention aslong as it departs from essential characteristics of the presentinvention.

In addition, some of components may not be indispensable componentsperforming essential functions of the present invention, but beselective components improving only performance thereof. The presentinvention may also be implemented only by a structure including theindispensable components except for the selective components, and thestructure including only the indispensable components is also includedin the scope of the present invention.

The image partition unit 100 partitions an input image into at least oneblock. The input image may have various shapes and sizes such as apicture, a slice, a tile, and a segment. A block means a coding unit(CU), a prediction unit (PU), or a transform unit (TU). The partitioningis performed based on at least one of a quadtree and a binary tree. Thequad tree is a method of dividing an upper layer block into smallerlower layer blocks with a width and a height that are half the upperlayer block. The binary tree is a method of dividing an upper layerblock into lower layer blocks, either width or height of which is halfthe upper layer block. Through the binary tree partitioning describedabove, square or non-square blocks can be generated.

Hereinafter, in the embodiment of the present invention, a coding unitis used as a basic unit for performing coding, or as a basic unit forperforming decoding.

A predicting unit 102 and 103 is divided into an inter predicting unit103 for performing inter prediction and an intra predicting unit 102 forperforming intra prediction. Whether to perform inter prediction orintra prediction is determined first for each prediction unit which isthe basic unit for prediction, and specific information (e.g., intraprediction mode, motion vector, reference picture, etc.) associated witheach of the prediction methods are also determined. The basic unit bywhich prediction is performed can differ from the basic unit by which aprediction method is determined or the basic unit by which detailedinformation for prediction is determined. For example, determination ofa prediction method and a prediction mode is performed a per predictionunit basis but execution of prediction is performed on a per transformunit basis.

A residual value (residual block) between a generated prediction blockwhich is the block generated through prediction and an original block isinput to the transformation unit 105. In addition, prediction modeinformation, motion vector information, and the like used for predictionare encoded by the entropy encoding unit 107, and the coded informationand the residual value are transmitted together to a decoder. When aparticular coding mode is used, the original block can be coded as it iswithout generating a prediction block by using the prediction units 102and 103, and the resulting coded block can be transmitted to thedecoder.

The intra prediction unit 102 determines an intra prediction mode usedfor performing prediction on a current block and generates one ormultiple prediction blocks using reference pixels according to thedetermined intra prediction mode. When the prediction mode of aneighboring block of the current block to undergo intra prediction is aprediction mode for inter prediction, a reference pixel included in theneighboring block which has undergone inter prediction is replaced witha reference pixel included in another neighboring block which hasundergone intra prediction. That is, when one or more reference pixelswithin a specific neighboring lock are not available, information onthese reference pixels are replaced with information on at least onereference pixel of available reference pixels.

For intra prediction, there are directional prediction modes andnon-directional prediction modes. In the directional prediction modes,reference pixel information selected according to direction informationis used for prediction. In contrast, in the non-directional predictionmodes, direction information is not used for prediction. The mode forpredicting luminance information can be the same or be different fromthe mode for predicting chrominance information. For prediction ofchrominance information, intra prediction mode information which hasbeen used for prediction of luminance information, or predictedluminance signal information can be used.

The intra prediction unit 102 includes an adaptive Intra smoothing (AIS)filter, a reference pixel interpolator, and a DC filter. The AIS filteris a filter for filtering the reference pixels of the current block.Whether to apply a filter is adaptively determined depending on theprediction mode of a current prediction unit. When the prediction modeof the current block is a mode in which AIS filtering is not performed,the AIS filter is not used.

When the intra prediction mode of a prediction unit is a prediction modein which intra prediction is performed by using pixel values generatedthrough interpolation of reference pixels, the reference pixelinterpolator of the intra prediction unit 102 generates reference pixelsfor sub-pixel positions by interpolating the reference pixels. Thereference pixels are not interpolated when the prediction mode of thecurrent prediction unit is a prediction mode in which a prediction blockis generated without interpolation of reference pixels. The DC filtergenerates a prediction block by filtering reference pixels when theprediction mode of the current block is DC mode.

A residual block including residual information that is a differencevalue between the prediction block generated by the prediction unit 102and 103 and the original block can be generated. The generated residualblock is input to the transformation unit 105 so as to be transformed.

FIG. 2 is a diagram illustrating an example of an intra prediction mode.Referring to FIG. 2 , there are a total of 35 modes for intraprediction. Mode 0 represents the planar mode, Mode 1 represents the DCmode, and Mode 2 through Mode 34 represent angular modes.

FIG. 3 is a diagram illustrating the planar mode. The predicted pixelvalue (hereinafter, referred to as prediction pixel) of a first pixel P1within the current block is generated by interpolating the pixel valueof the reconstructed pixel at the same position in the Y axis as thefirst pixel P1 and the pixel value of the reconstructed pixel T at theupper right position with respect to the current block. Similarly, thepredicted pixel value (hereinafter, referred to as prediction pixel) ofa second pixel P2 within the current block is generated by performinglinear interpolation using the reconstructed pixel at the same positionin the X axis as the second pixel P2 and the reconstructed pixel L atthe lower left position with respect to the current block. A valueobtained by averaging two prediction pixels P1 and P2 is a finalprediction pixel. In the planar mode, a prediction block of the currentblock is generated by deriving prediction pixels in the same manner asdescribed above.

FIG. 4 is a diagram illustrating the DC mode. The average of thereconstructed pixels around the current block is first calculated. Theaverage is used as the predicted pixel value of each pixel within thecurrent block.

FIG. 5 is a diagram illustrating an example of a method of generating aprediction block using Mode 10 (horizontal mode) and Mode 26 (verticalmode). In the case of using Mode 10, the pixel values of the referencepixels adjacent to the left side of the current block are copied totheir right-side pixels within the current block, thereby generating aprediction block of the current block. Similarly, in the case of usingMode 26, the pixel values of the reference pixels adjacent to the upperside of the current block are copied downward to their lower-sidepixels, thereby generating a prediction block of the current block.

Referring to FIG. 1 , the inter prediction unit 103 generates aprediction unit on the basis of information on the previous picture, thesubsequent picture, or both of a current picture. In some cases, theprediction unit is generated on the basis of information on a localregion within a picture. The inter prediction unit 103 includes areference picture interpolator, a motion prediction unit, and a motioncompensation unit.

The reference picture interpolator receives reference pictureinformation from the memory unit 112 and generates pixel information oninteger pixels or sub-pixels from a reference picture. For a luminancepixel, a DCT-based interpolation 8-tap filter having different filtercoefficients is used to generate pixel information on sub-pixels on aper ¼-pixel basis. For a chrominance pixel, a DCT-based interpolation4-tap filter having different filter coefficients is used to generatepixel information on sub-pixels on a per ⅛-pixel basis.

The motion prediction unit performs motion prediction on the basis of areference picture that is generated through interpolation by thereference picture interpolator. Various methods such as FullSearch-based Block Matching Algorithm (FBMA), Three Step Search (TSS),and New Three-Step Search Algorithm (NTS) can be used to calculatemotion vectors.

The motion vector has a motion vector value per ½-pixel or ¼-pixel,which is generated on the basis of sub-pixels generated through theinterpolation. The motion prediction unit performs prediction on acurrent prediction unit by switching motion prediction methods. Variousmotion prediction methods such as a skip method, a merge method, and anadvanced motion vector prediction (AMVP) method can be used.

The subtractor 104 generates a residual block of the current block bycalculating a difference between the current block to be coded and theprediction block generated by the intra prediction unit 102 or the interprediction unit 103.

The transformation unit 105 transforms the residual block includingresidual data by using a conversion transform method such as DCT, DST,Karhunen Loeve Transform (KLT), or the like. In this case, a transformmethod to be used is determined depending on the intra prediction modeof the prediction unit that has been used to generate the residualblock. For example, depending on the intra prediction mode, DCT and DSTmay be used for the horizontal direction and the vertical direction,respectively.

The quantization unit 106 quantizes the values transformed into thefrequency domain by the transformation unit 105. The quantizationcoefficient varies depending on the block or the importance of theimage. The values calculated by the quantization unit 106 are fed to thedequantization unit 108 and the entropy encoding unit 107.

The transformation unit 105, the quantization unit 106, or both areincluded in the image encoding apparatus 100. That is, the imageencoding apparatus 100 encodes the residual block by performingtransform, quantization, or both on the residual data of the residualblock, or by skipping both the transform and the quantization. A blockthat is fed to the entropy encoding unit 107 is generally referred to asa transform block even though either the transform or the quantizationis not performed in the image encoding apparatus 100, or neither thetransform nor the quantization is performed in the image encodingapparatus 100. The entropy encoding unit 107 entropy-encodes input data.For entropy encoding, various encoding methods such asexponential-Golomb, context-adaptive variable length coding (CAVLC) andcontext-adaptive binary arithmetic coding (CABAC) can be used.

The entropy encoding unit 107 receives from the prediction unit 102 and103 various kinds of information such as residual coefficientinformation and block type information for each coding unit, predictionmode information, partition unit information, prediction unitinformation, transmission unit information, motion vector information,reference frame information, block interpolation information, filteringinformation, and the like, and encodes the received information. Atransform block coefficient is determined for each partial block withina transform block. The entropy encoding unit 107 encodes a coefficienthaving a value other than 0, a coefficient having an absolute valuegreater than 1 or 2, and various kinds of flag indicating the signs ofthe coefficients. The coefficient that is not to be encoded with only aflag can be encoded with the absolute value of the difference between anactual transform block coefficient and a coefficient that is encodedwith a flag. The dequantization unit 108 dequantizes the valuesquantized by the quantization unit 106 and the inverse transformationunit 109 inversely transforms the values transformed by thetransformation unit 105. A residual generated by the dequantization unit108 and the inverse transformation unit 109 is added to a predictionunit that is generated through operations of the motion estimator, themotion compensation unit, and the intra prediction unit 102 of theprediction unit 102 and 103. Thus, a reconstructed block is generated.The adder 110 generates a reconstruction block by summing a predictionblock generated by the predicting unit 102 and 103 and a residual blockgenerated by the inverse transform unit 109.

The filter unit 111 include at least one of a deblocking filter, anoffset compensation unit, and an adaptive loop filter (ALF).

The deblocking filter removes block artifacts caused by the boundarybetween blocks within a reconstructed picture. When determining whetherto perform deblocking, pixels within several rows or columns within ablock are used for the determination. When it is determined that adeblocking filter is to be applied to a block, a strong filter or a weakfilter may be used according to a deblocking filtering strengthrequired. When performing horizontal filtering and vertical filteringwhile using a deblocking filter, the vertical filtering and thehorizontal filtering can be performed in parallel.

The offset compensation unit compensates the offset between thedeblocked image and the original image on a per pixel basis. In order toperform offset compensation for a specific picture, pixels included inthe specific picture are divided into a predetermined number of regions,a region to undergo offset compensation is then determined, and anoffset compensation is performed on the determined region.Alternatively, an offset compensation may be applied according to edgeinformation of each pixel.

Adaptive loop filtering (ALF) can be performed on the basis of acomparison result between the filtered reconstructed image and theoriginal image. Filtering can be differently performed depending on apixel group. That is, pixels within an image are divided into apredetermined number of pixel groups, filters to be used for therespective pixel groups are determined, and filtering is differentlyperformed on each pixel group. The information indicating whether toapply the ALF is transmitted per coding unit (CU). The shape and thefilter coefficient of the ALF filter to be used may differ for eachblock. Alternatively, the same type (fixed form) of ALF filter may beused, regardless of the characteristics of target blocks to be filtered.

The memory unit 112 stores reconstructed blocks or reconstructedpictures output from the filter unit 111 and the reconstructed blocks orpictures stored therein are fed to the prediction unit 102 and 103 whenthe inter prediction is performed.

FIG. 6 is a block diagram illustrating an image encoding apparatus 600according to an embodiment of the present invention.

Referring to FIG. 6 , the image decoding apparatus 600 includes anentropy decoding unit 601, a dequantization unit 602, an inversetransformation unit 603, an adder 604, a filter unit 605, a memory unit606, and a prediction unit 607 and 608.

When an image bitstream generated by the image encoding apparatus 100 isinput to the image decoding apparatus 600, the input bitstream isdecoded according to the reverse operation procedure by the imagedecoding apparatus 600.

The entropy decoding unit 601 perform entropy decoding in the reverseprocedure to the procedure performed by the entropy encoding unit 107 ofthe image encoding apparatus 100. For example, various methods such asexponential-Golomb, context-adaptive variable length coding (CAVLC) andcontext-adaptive binary arithmetic coding (CABAC) can be used accordingto the method used in the image encoding apparatus. A transform blockcoefficient is determined for each partial block within a transformblock. The entropy decoding unit 601 decodes a coefficient having avalue other than 0, a coefficient having an absolute value greater than1 or 2, and various kinds of flags based the signs of the coefficients.A coefficient that is not represented only by the flag is decoded bysumming a coefficient represented by a flag and a signaled coefficient.

The entropy decoding unit 601 can decode information associated theintra prediction and the inter prediction performed in the encoder. Thedequantization unit 602 performs dequantization on the quantizedtransform block to generate a transform block. The dequantization unit602 operates in the substantially same manner as the dequantization unit108 of FIG. 1 .

The inverse transformation unit 603 performs inverse transform on thetransform block to generate a residual block. In this case, thetransform method is determined depending on information indicating aprediction method (whether it is inter prediction or intra prediction),the size and/or shape of a block, an intra prediction mode, and thelike. The inverse transformation unit 603 operates in the substantiallysame manner as the inverse transformation unit 109 of FIG. 1 .

The adder 604 generates a reconstruction block by summing a predictionblock generated by the predicting unit 607 and 608 and a residual blockgenerated by the inverse transformation unit 603. The adder 604 operatesin the substantially same manner as the adder 110 of FIG. 1 .

The filter unit 605 reduces various types of noise occurring in thereconstructed blocks.

The filter unit 605 includes a deblocking filter, an offset compensationunit, and an ALF.

Information on whether or not a deblocking filter has been applied tothe corresponding block or picture is received from the image encodingapparatus 100. When a deblocking filter is applied, information onwhether a strong filter or a weak filter is applied is received from theimage encoding apparatus. The deblocking filter of the image decodingapparatus 600 receives information on the used deblocking filter fromthe image encoding apparatus 100. The deblocking filter of the imagedecoding apparatus 600 performs deblocking filtering on the target blockon the basis of the received information.

The offset compensation unit performs offset compensation on areconstructed image on the basis of the offset compensation type and theoffset value that have been used for image encoding.

The ALF is applied or not applied to a coding unit according to ALFapplication information indicating whether an ALF is applied duringencoding, ALF coefficient information, and the like, which are providedby the image encoding apparatus 100. Such ALF information is included ina specific parameter set. The filter unit 605 operates in thesubstantially same manner as the filter unit 111 of FIG. 1 .

The memory unit 606 stores the reconstructed block generated by theadder 604. The memory unit 606 operates in the substantially same manneras the memory unit 112 of FIG. 1 .

The prediction unit 607 and 608 generates a prediction block on thebasis of information associated with prediction block generationprovided by the entropy decoding unit 601 and information on thepreviously decoded block or picture provided by the memory unit 606.

The prediction unit 607 and 608 includes an intra prediction unit 607and an inter prediction unit 608. Although not illustrated in thedrawings, the prediction unit 607 and 608 may further include aprediction-unit determination unit. The prediction-unit determinationunit receives various information such as prediction unit informationinput from the entropy decoding unit 601, prediction mode information ofthe intra prediction method, and motion prediction information of theinter prediction method. The prediction-unit determination unitidentifies the prediction unit of the current coding unit and determineswhether the prediction unit is to be inter-predicted or intra-predicted.The inter prediction unit 608 performs inter prediction on the currentprediction unit by using information required for inter prediction ofthe current prediction unit, which is provided by the image encodingapparatus 100, on the basis of information included within the previouspicture or the subsequent picture to the current picture in which thepresent prediction unit is included. Alternatively, inter prediction maybe performed on the basis of information of a partial region of apreviously reconstructed region within the current picture in which thecurrent prediction unit is included.

In order to perform the inter prediction, for each coding unit, it isdetermined which motion prediction mode from among a skip mode, a mergemode, and an AMVP mode is used as a motion prediction mode of aprediction unit included in a corresponding one of the coding units.

The intra prediction unit 607 generates a prediction block usingreconstructed pixels located near the current block to be encoded.

The intra prediction unit 607 includes an adaptive Intra smoothing (AIS)filter, a reference pixel interpolator, and a DC filter. The AIS filteris a filter for filtering the reference pixels of the current block.Whether to apply a filter is adaptively determined depending on theprediction mode of a current prediction unit. AIS filtering is performedon reference pixels of the current block by using a prediction mode of aprediction unit and AIS filter information provided by the imageencoding apparatus 100. When the prediction mode of the current block isa mode in which the AIS filtering is not performed, an AIS filter is notused.

When a prediction mode of a prediction unit is a mode in which intraprediction is performed on the basis of a pixel value obtained byinterpolating reference pixels, the reference pixel interpolator of theintra prediction unit 607 interpolates the reference pixels, therebygenerating “reference pixels at sub-pixel positions” (hereinafter,referred to as sub-pixel-position reference pixels). The generatedsub-pixel-position reference pixels can be used as prediction pixels ofthe pixels within the current block. When the prediction mode of thecurrent prediction unit is a prediction mode in which a prediction blockis generated without interpolating reference pixels, the referencepixels are not interpolated. The DC filter generates a prediction blockby filtering reference pixels when the prediction mode of the currentblock is DC mode.

The intra prediction unit 607 operates substantially in the same manneras the intra prediction unit 102 of FIG. 1 .

The inter prediction unit 608 generates an inter prediction block byusing the reference picture and the motion information stored in thememory unit 606. The inter prediction unit 608 operates substantially inthe same manner as the inter prediction unit 103 of FIG. 1 .

The present invention particularly relates to intra prediction.Hereinafter, various embodiments of the present invention will bedescribed in greater detail below with reference to the accompanyingdrawings.

<Interpolation for Intra Prediction>

FIGS. 7A and 7B are diagrams illustrating a method of generating anintra prediction pixel by using an interpolation scheme; Assuming thatthe prediction angle of a mode m (mode number is m), which is one of theintra prediction modes illustrated in FIG. 2 , is the same as shown inFIG. 7A, when intra prediction is performed using the mode m, areference pixel X to be used for prediction is not located at an integerpixel position. Therefore, interpolation is performed using referencepixels A and B located at integer pixel positions on the left side andthe right side of the reference pixel X, and the reference pixel X at asub-pixel position is generated. The generated reference pixel X is usedas a prediction pixel at a position P within the current block.

FIG. 7B is a diagram illustrating a relationship among the pixels X, A,and B. Referring to FIG. 7B, the distance between the pixels X and A isS1, and the distance between the pixels B and X is S2. The pixel X canbe derived using one of various interpolation schemes, selecteddepending on the ratio of the distances S1 and S2. Various interpolationschemes such as linear interpolation, cubic convolution interpolation,and B-spline interpolation can be used for interpolation.

There are various methods of enabling the image decoding apparatus 600to be aware of which interpolation scheme is used from among multipleinterpolation schemes or which interpolation coefficient set is used. Afirst method is a method in which the image encoding apparatus 100transmits index information indicating which interpolation scheme amongmultiple available interpolation schemes is used to the image decodingapparatus 600. In this case, the image encoding apparatus 100 can setthe index information indicating the interpolation scheme by using ablock header or an upper layer header. Here, setting the indexinformation by using the upper layer header means that the header of aunit larger than a block, for example, a slice segment header, a pictureparameter set, or a sequence parameter set is used. The indexinformation indicating the interpolation scheme included in the upperlayer header is encoded by the image encoding apparatus 100 and theencoded index information is transmitted to the image decoding apparatus600.

Alternatively, the image encoding apparatus 100 and the image decodingapparatus 600 store the same predetermined multiple interpolationcoefficient sets, and interpolation coefficient index informationindicating which set is selected and used for encoding is notified tothe image decoding apparatus 600 via the upper layer header.

Alternatively, instead of the method in which the image encodingapparatus 100 transmits index information indicating which interpolationscheme is used or interpolation coefficient index information indicatingwhich interpolation coefficient set is used to the image decodingapparatus 600, a different method can be used in which the imageencoding apparatus 100 and the image decoding apparatus 600 derive thesame interpolation coefficient set implicitly.

Specifically, the image encoding apparatus 100 and the image decodingapparatus 600 can derive the same interpolation coefficient set in thesame manner by using previously reconstructed pixels. For example, oneinterpolation filter is used to increase R reference pixels that arereconstructed pixels by K times (i.e., increased to R×K referencepixels) where K is an arbitrary real number, or decrease the R referencepixels by 1/K times. Then, the original R reference pixels are restoredthrough the reverse process using the same interpolation filter. Theoptimal interpolation filter is determined according to the differencesbetween the values of the R reconstructed reference pixels and thevalues of the original reference pixels.

FIG. 8 is a diagram illustrating another method of selecting aninterpolation scheme and/or an interpolation coefficient set in animplicit manner by the image encoding apparatus 100 or the imagedecoding apparatus 600. Referring to FIG. 8 , a 4×4 block including apixel P corresponds to a current block to be decoded through intraprediction. Multiple reference pixel lines, which are located around thecurrent block and are composed of reconstructed pixels, are used fordetermination of an interpolation scheme or an interpolationcoefficient. As illustrated in FIG. 8 , each reference pixel lineincludes a predetermined number of pixels arranged in a line extendingin a horizontal direction and a predetermined number of pixels arrangedin a line extending in a vertical direction. Alternatively, thereference pixel line may be composed of a predetermined number of pixelsarranged in a line extending in the horizontal direction or apredetermined number of pixels arranged in a line extending in thevertical direction.

Referring to FIG. 8 , the pixels within Reference pixel line 0 arepredicted using the pixels within Reference pixel line 1. In this case,Mode N, which is one of the directional modes and is the same as theintra prediction mode of the current block, is used for prediction. Forexample, since a reference pixel X which is a prediction pixel of apixel R included in Reference pixel line 0 is not an integer-positionpixel, the reference pixel X is derived by interpolating twointeger-position reference pixels as shown in FIGS. 7A and 7B. In thiscase, a specific interpolation scheme and a specific interpolationcoefficient are used.

In this way, the predicted values of the pixels within Reference pixelline 0 are generated, the difference values between the predicted valuesand the corresponding original pixel values are calculated, and thedifference values are summed. The image encoding apparatus 100 or theimage decoding apparatus 600 repeats the above described processes usingavailable interpolation schemes and interpolation coefficients andselects an interpolation scheme and or an interpolation coefficient withwhich the sum of the residuals is least.

The above-described interpolation is performed by the reference pixelinterpolators that are respectively included in the intra predictionunit 102 of the image encoding apparatus 100 and the intra predictionunit 607 of the image decoding apparatus 600.

FIG. 9 is a flowchart illustrating a process in which the image encodingapparatus 100 selects the optimum intra prediction mode. In this case,it is assumed that the interpolation scheme is set by using a blockheader or an upper layer header.

Referring to FIG. 9 , a variable m indicating the mode number of anintra prediction mode is initialized to 0 (i.e., m=0), a variableCOST_BEST for storing an optimal cost value is initialized to themaximum cost value MAX_VALUE (i.e., COST_BEST=MAX_VALUE) (S901). WhereMAX_VALUE is the maximum value that can be stored in the variableCOST_BEST and is a very large value that cannot actually be calculatedin the cost calculation. The total number of predetermined intraprediction modes is set in the variable M (S901). BEST_INTRA_MODEindicating the optimal intra prediction mode for the current block isinitialized to 0 (i.e., BEST_INTRA_MODE=0) (S901.

Next, the interpolation position corresponding to each pixel positionwithin a block is searched for according to the intra prediction mode m,an interpolation value is generated using a predetermined interpolationscheme or one of multiple interpolation schemes set in the upper layerheader, and a prediction block is generated (S902). Next, COST_m, whichis a cost value corresponding to m, is calculated using the generatedprediction block (S903). Here, COST_m is calculated by using the numberof bits required to encode the intra prediction mode, and the differencebetween the prediction block and the current block. When COST_m is lessthan or equal to COST_BEST (S904), the m is stored in BEST_INTRA_MODE,which is a variable for storing the optimal intra prediction mode,cost_m is stored in the variable COST_BEST, and m is increased by 1(S905). When COST_m is greater than COST_BEST, only m is increased by 1(S906). Finally, when m reaches the maximum number of intra predictionmodes, the process ends. When m is less than the maximum number of intraprediction modes, the process returns to S902, and S902 and thesubsequent steps are repeatedly performed. When as the interpolationscheme, an interpolation scheme that is preset in the image encodingapparatus 100 or the image decoding apparatus 600 is used, S1 and S2 areset using the methods illustrated in FIGS. 7 and 8 , and the pixel X isgenerated by using the preset interpolation scheme. Every predictionpixel (i.e., the predicted value of every pixel) within a predictionblock is generated using the same method. Alternatively, when multipleinterpolation schemes are used, S902 is performed in a different manner.

Each prediction block is adaptively generated by using a differentinterpolation scheme. In this case, S902 among the multiple steps shownin FIG. 9 is changed.

FIG. 10 is a flowchart illustrating a process in which the imageencoding apparatus 100 selects an interpolation scheme from amongmultiple interpolation schemes.

Referring to FIG. 10 , the image encoding apparatus 100 initializes avariable i representing an interpolation scheme index to 0 (i.e., i=0),and initializes a variable COST_BEST_i for storing an optimal cost valueto the maximum cost value MAX_VALUE (i.e., COST_BEST_i=MAX_VALUE). WhereMAX_VALUE is the maximum value that can be stored in the variableCOST_BEST and is a very large value that cannot actually be producedthrough the cost calculation. The variable i is set to the total numberof available interpolation schemes. BEST_INTERPOLATION, which is avariable for storing the optimal interpolation scheme used for thecurrent block, is initialized to 0 (S1001). Next, interpolation valuescorresponding to respective pixel positions of a prediction block aregenerated according to the interpolation scheme index i, and aprediction block is generated (S1002). Next, COST_i, which is a costvalue corresponding to i, is calculated using the generated predictionblock (S1003). Here, COST_i is calculated by using the number of bitsrequired to encode the interpolation scheme index and the differencebetween the prediction block and the current block. When COST_i is lessthan or equal to COST_BEST_i (S1004), i is stored in BEST_INTERPOLATION,which is a variable for storing the optimal interpolation scheme, cost_iis stored in the variable COST_BEST_i, and i is increased by 1 (S1005).When COST_i is greater than COST_BEST_i, only i is increased by 1(S1006). Finally, when i reaches the maximum number of availableinterpolation schemes, the process ends. When not, the process returnsto S1002, and S1002 and the subsequent steps are repeatedly performed.When this scheme is used, the number of bits required to encode theinterpolation scheme index is added to the number of bits required toencode the intra prediction mode in Step S903 illustrated in FIG. 9 tocalculate the cost COST_m.

FIG. 11 is a flowchart illustrating a process of encoding interpolationscheme index information when one of multiple interpolation schemes isadaptively selected for each prediction block by the image encodingapparatus 100. First, information of whether an intra prediction mode ispredicted is encoded for each prediction block (S1101). Next, whetherprediction has been performed is determined (S1102). When it isdetermined that prediction has been performed, an index indicating whichcandidate among intra prediction mode candidates derived fromneighboring blocks is selected is encoded (S1103). Otherwise, theremaining modes except for the intra prediction mode candidates derivedfrom the neighboring blocks are re-arranged, and the currently selectedintra prediction mode is binarized and encoded (S1104). Next, theinterpolation scheme index indicating the used interpolation scheme isencoded (S1105) and the process ends.

FIG. 12 is a flowchart illustrating a process in which the imagedecoding apparatus 600 decodes the interpolation scheme indexinformation. First, information of whether intra prediction mode ispredicted is decoded for each prediction block (S1201). Next, it ischecked whether prediction has been performed (S1202). When it isdetermined that prediction has been performed, an index indicating whichcandidate among intra prediction mode candidates derived fromneighboring blocks is selected is decoded (S1203). Otherwise, theremaining modes except for the intra prediction mode candidates derivedfrom the neighboring blocks are re-arranged, and the currently selectedintra prediction mode is decoded (S2104). Next, the interpolation schemeindex indicating the interpolation scheme used the encoder is decoded(S1205) and the process ends.

<Derivation of Intra Prediction Pixel Using Multiple Reference PixelLines>

Hereinafter, a method of deriving an intra prediction pixel usingmultiple reference pixel lines, according to another embodiment of thepresent invention, will be described.

FIG. 13 is a diagram illustrating a process of deriving an intraprediction pixel by using multiple reference pixel lines, according toan embodiment of the present invention;

Conventionally, one reference pixel line is used for intra prediction.For example, a reference pixel line 0 illustrated in FIG. 13 is a singlereference pixel line used for conventional intra prediction. Thereference pixel line 0 includes a predetermined number of referencepixels adjacent to the upper boundary of the current block and apredetermined number of reference pixels adjacent to the left boundaryof the current block. The present invention can improve the accuracy ofintra prediction by deriving a prediction pixel or a prediction block byusing various reference pixel lines and reference pixels belonging tothe reference pixel lines. The present embodiment can be similarlyperformed by the intra prediction unit 102 of the image encodingapparatus 100 and the intra prediction unit 607 of the image decodingapparatus 600.

In the following description, it is assumed that a total of threereference pixel lines are used. However, an arbitrary number ofreference pixel lines can be used instead of three. Here, the number Nof reference pixel lines is included in a block header or an upper layerheader so as to be notified to the image decoding apparatus 600.Alternatively, it is also possible that the image encoding apparatus 100and the image decoding apparatus 600 use predetermined N reference pixellines without encoding the number N of reference pixel lines.

Referring to FIG. 13 , a 4×4 block including a pixel P corresponds to acurrent block to be encoded or decoded through intra prediction. Threereference pixel lines 0, 1 and 2 are located around the current block.

When the intra prediction mode of the current block is a directionalmode with a mode number m, pixels X, Y, and Z within the three referencelines 0, 1, and 2, respectively, can be used as prediction pixels of thepixel P. In this case, it is possible to generate a prediction blockusing each of the three reference pixel lines and determine an optimumreference pixel line. Reference pixel line index information indicatingthe determined optimum reference pixel line is encoded by the imageencoding apparatus 100. For example, as shown in FIG. 13 , a lower indexnumber is assigned as a reference pixel line index for a reference pixelline closer to the current block.

FIG. 14 is a flowchart illustrating a process of deriving an intraprediction pixel value, according to an embodiment of the presentinvention. Referring to FIG. 14 , at least one reference pixel line tobe used for intra prediction of a current block is selected from amongmultiple reference pixel lines (S1301). The multiple reference pixellines are present in the same image as the current block to be decodedusing intra prediction. At least one reference pixel line that isselected for intra prediction of the current block is indicated by areference pixel line index described above. Alternatively, at least onereference pixel line to be used for intra prediction of the currentblock may be selected by the image encoding apparatus 100 and the imagedecoding apparatus 600 in an implicit manner to be described later.

At least one reference pixel line may be selected per prediction block.This case will be described later with reference to FIG. 15 .Alternatively, it may be adaptively selected for each pixel within theprediction block. This case will be described later with reference toFIGS. 18 and 19 .

The image encoding apparatus 100 or the image decoding apparatus 600obtains a predicted value of one pixel within the current block on thebasis of at least one pixel value included in the one or more selectedreference pixel lines (S1303). The image encoding apparatus 100 or theimage decoding apparatus 600 may Step S1301, Step S1303, or both toderive a prediction block of the current block.

FIG. 15 is a flowchart illustrating a process of adaptively determininga reference pixel line to be used for intra prediction for eachprediction block. In this case, Step S902 shown in FIG. 9 is replacedwith the steps shown in FIG. 15 .

Referring to FIG. 15 , a variable n for a reference pixel line index isinitialized to 0 (i.e., n=0), and a variable COST_BEST_n for storing anoptimal cost value is initialized to MAX_VALUE (i.e.,COST_BEST_n=MAX_VALUE). Here, MAX_VALUE is the maximum value that can bestored in the variable COST_BEST and is a very large value that cannotactually be produced through the cost calculation. A variable N is setto the total number of preset reference pixel lines. BEST_n which is areference pixel line index representing the optimal reference pixel linefor the current block is initialized to 0 (S1401). Next, aninterpolation position corresponding to each pixel position within aprediction block is identified according to the reference pixel lineindex n, and a prediction block is generated (S1402). Next, COST_n,which is a cost value corresponding to n, is calculated using thegenerated prediction block (S1403). Here, COST_n is calculated by usingthe number of bits required to encode the reference pixel line index andthe difference between the prediction block and the current block. WhenCOST_n is less than or equal to COST_BEST_n (S1404), n is stored inBEST_n, which is a variable for storing the optimum reference pixelline, Cost_n is stored in the variable COST_BEST_n, and n is increasedby 1 (S1405). When COST_n is greater than COST_BEST_n, n is increased by1 (S1406). Finally, when i reaches the maximum number of reference pixellines, the process ends. Otherwise, the process returns to S1402. Thus,S1002 and the subsequent steps are performed.

FIG. 16 is a flowchart illustrating a process in which reference pixelline index information indicating a selected reference pixel line isencoded by the image encoding apparatus 100 when a reference pixel lineis adaptively selected for each prediction block. First, information ofwhether prediction for an intra prediction mode is performed for eachprediction block is encoded (S1501). Next, it is checked whether theprediction has been performed (S1502). When it is determined that theprediction has been performed, an index indicating which candidate amongintra prediction mode candidates derived from neighboring blocks isselected is encoded (S1503). Otherwise, the remaining modes except forthe intra prediction mode candidates derived from the neighboring blocksare re-arranged, and the currently selected intra prediction mode isbinarized and encoded (S1504). Next, the reference pixel line indexindicating the used reference pixel line is encoded (S1505) and theprocess ends.

FIG. 17 is a flowchart illustrating a process in which reference pixelline index information indicating a selected reference pixel line isdecoded by the image decoding apparatus 600 when the reference pixelline is adaptively selected for each prediction block. First,information of whether prediction for an intra prediction mode isperformed is decoded for each prediction block (S1601). Next, it ischecked whether the prediction has been performed (S1602). When it isdetermined that the prediction has been performed, an index indicatingwhich candidate among intra prediction mode candidates derived fromneighboring blocks is selected is decoded (S1603). Otherwise, theremaining modes except for the intra prediction mode candidates derivedfrom the neighboring blocks are re-arranged, and the currently selectedintra prediction mode is decoded (S1604). Next, the used reference pixelline index is decoded (S1605) and the process ends.

Next, with reference to FIGS. 18 and 19 , a method of adaptivelydetermining a reference pixel line for each pixel position within aprediction block without transmitting a reference pixel line index willbe described.

For each pixel within a prediction block, the precision of the positionsof prediction pixels obtained by interpolating reference pixels within areference pixel line may differ. Therefore, among the prediction pixelsobtained through interpolation within each reference pixel line, aprediction pixel closest to an integer pixel position is selected as aprediction pixel of a current pixel P. In this case, the above processcan be performed on N reference pixel lines that are predetermined.

When there are multiple prediction pixels at integer pixel positions, aprediction pixel close to the current block is selected as the finalprediction pixel.

FIG. 19 is a reference diagram illustrating a method of adaptivelyselecting a reference pixel line without transmitting a reference pixelline index. As shown in FIG. 19 , a prediction pixel line is adaptivelyselected for each prediction block, with priority given to a line havinga larger number of pixels located at inter positions. Assuming that theprecision and the frequency of the interpolated pixels used to generatethe prediction block by using each of the reference pixel lines are thesame as illustrated in FIG. 19 , a line may be selected while weightingaccording to the precision of each pixel position.

Referring to FIG. 19 , when generating a prediction block using a line1, five prediction pixels at integer pixel positions, three predictionpixels at ½-pixel positions, four prediction pixels at ¼-pixelpositions, two prediction pixels at ⅛-pixel positions, one predictionpixel at a 1/16-pixel position, and one prediction pixel at a 1/32-pixelposition are selected. Accordingly, there are a total of 16 predictionpixels within the prediction block. This similarly applies to areference pixel line 2 and a reference pixel line 3.

When priority is given only to integer pixel positions, the line 1having the largest number of integer positions is selected as thereference pixel line. Alternatively, weights may be given to respectivepositions, the weighted sums may be calculated for each line, and theline having the largest calculated value may be selected as thereference pixel line. Alternatively, weights may be given to respectivelines, the weighted sums may be calculated for each line, and the linehaving the largest calculated value may be selected as the referencepixel line. Alternatively, weights may be given to respective positionsand respective lines, the weighted sums may be calculated for each line,and the line having the largest calculation value may be selected as thereference pixel line.

As another embodiment, weights are given to respective lines, and aprediction block is generated by using the weighted sum as a pixelvalue. For example, when pixel values of the X, Y, and Z positions inFIG. 13 exist, a larger weight is given to a pixel closer to the currentblock, and the weighted sum is selected as a prediction pixel for aposition P. Alternatively, a larger weight may be given to a pixelcloser to an inter pixel position, and the weighted sum may be selectedas a prediction pixel at for the position P. Alternatively, in additionto the method of deriving a prediction pixel using the weighted sum,that is, the weighted average, the predicted value of a pixel can bederived by using an arithmetic average, a median value, or the like.

Alternatively, from among N reference pixel lines, one reference pixelline that is indicated by an encoded reference pixel line index isexcluded, and N−1 reference pixel lines may be used. For example, whenthe reference pixel line index is set to 1, N−1 lines excluding the line1 are used. In this case, when a predicted value is interpolated byusing a mode with a mode number m, a higher priority is given to aposition closer to an integer pixel position, or the priorities aregiven differently according to the precision of the interpolatedpositions. Alternatively, it is possible to generate a predicted valueon a per pixel basis, by using reference pixels selected from arbitraryreference pixel lines except for the line 1, in accordance with apredetermined priority order.

Alternatively, a case is also possible in which the image encodingapparatus 100 encodes information indicating whether a method ofdirectly encoding a reference pixel line index into a block header or anupper layer header is used or a method of not encoding a reference pixelline index is used, and transmits the encoded information to the imagedecoding apparatus 600.

<Smoothing Between Prediction Block and Reference Pixel Line>

Hereinafter, smoothing between the prediction block and the referencepixel line will be described as another embodiment of the presentinvention.

When a prediction block is derived by using a predetermined pixel orpredetermined multiple pixels within a reference pixel line, there maybe a discontinuity between reference pixel line(s) that is (or are) thatused for the derivation of the prediction block and the predictionblock, or between the prediction block and a region adjacent to theprediction block. In order to reduce the discontinuity, smoothing isperformed. A smoothing filter is a kind of low-pass filter.

The smoothing according to an embodiment of the present invention isperformed by both of the intra prediction unit 102 of the image encodingapparatus 100 and the intra prediction unit 607 of the image decodingapparatus 600.

FIG. 20 is a diagram illustrating a method of smoothing the boundarybetween a prediction block and a reference pixel line.

Hereinafter, a case where a mode corresponding to a 45° up-rightdirection is used for intra prediction will be described. In addition,it is assumed that a reference pixel line 1 is selected for intraprediction. In addition, in the following description, the smoothing isapplied to pixels A to E. However, this smoothing may similarly apply toother pixels.

In the example of FIG. 20 , since the intra prediction is performed inthe 45° up-right direction, the smoothing is performed in a 45°down-left direction which is opposite to the direction in which theintra prediction direction is performed. In this case, a region of aprediction block to which the smoothing is applied is determinedaccording to the size or type of the prediction block of the currentblock. In FIG. 20 , the pixels belonging to the half of the predictionblock undergo the smoothing. That is, the smoothing is applied only tothe shaded pixels located within the left half region. Alternatively, apredetermined region or a region corresponding to a predetermined ratiomay undergo the smoothing, depending the size of the prediction block orthe intra prediction mode. For example, only one quarter of theprediction block may undergo smoothing. Of the prediction block, apartial region corresponding to a different ratio may undergo thesmoothing.

In FIG. 20 , since the reference pixel line 1 is determined as thereference pixel line for intra prediction, a prediction pixel A may besmoothed using Equation 1 and a pixel D within the reference pixel line1.

A′=(w1×A+w2×D)/(w1+w2)  [Equation 1]

In Equation 1, A′, A, and D are respectively the value of the predictionpixel A which results from the smoothing, the initial value of theprediction pixel A before the smoothing, and the value of the referencepixel D. Further, w1 and w2 are weights applied to the prediction pixelA and the reference pixel D, respectively.

Further, the prediction pixel B is smoothed by using an equation similarto Equation 1 and the reference pixel D within the reference pixel line1.

The strength of the smoothing is determined depending on the distance.Since the distance between the prediction pixel A and the pixel D islonger than the distance between the prediction pixel B and the pixel D,smoothing is more strongly performed for the prediction pixel A and thereference pixel D than for the prediction pixel B and the referencepixel D. The stronger smoothing is performed by placing a larger weighton the reference pixel D.

The reference pixel line used for smoothing may differ from thereference pixel line used for intra prediction. A reference pixel linecloser to the prediction block may be used for smoothing. Referring toFIG. 20 , although the reference pixel line 1 is selected for intraprediction, the pixel used for smoothing of the prediction pixels A andB is not set to D but is set to C. In this case, the strength of thesmoothing is selected depending on the distance. For example, when theprediction pixel B is smoothed using the reference pixel C, the sameweight is applied to each pixel. When the prediction pixel A is smoothedusing the reference pixel C, a larger weight is placed on the pixel Cfor smoothing.

When performing smoothing, it can be bidirectionally performed. Forexample, in a case where the reference pixel line 0 is used forsmoothing, when the prediction pixel A is smoothed, the prediction pixelA is smoothed while applying different weights to the reference pixels Fand C. When smoothing the prediction pixel B, the prediction pixel B canbe smoothed while applying different weights to the reference pixels Fand C. At this time, since the line 1 is selected as the reference pixelline for intra prediction, the pixel G may be used for the up-rightdirection and the pixel C may be used for the down-left direction.

According to an embodiment of the present invention, the weights mayvary depending on the distances between the reference pixels and theprediction pixels. For example, when the prediction pixel A is smoothedusing the reference pixels F and C, since the distance between thepixels C and A is longer than the distance between the pixels F and A,smoothing is performed by placing a larger weight on the reference pixelC. It is also possible to perform smoothing using arbitrary lines and apredetermined method. It is also possible to encode information onwhether smoothing is performed per block. Alternatively, it can beencoded using an upper layer header. It is also possible that theencoder and the decoder perform the same operation under predeterminedconditions without encoding the information on whether smoothing isapplied or not. For example, it may be determined whether or notsmoothing is performed depending on which of the intra prediction modesis used. In the present embodiment, the strength of the smoothing isincreased as the distance between the prediction pixel and the referencepixel is increased. However, the opposite is also possible depending onthe characteristics of the image.

Next, with reference to FIG. 21 and FIGS. 22A to 22D, according to anembodiment of the present invention, a basic unit by which a referencepixel line is selected for intra prediction will be described.

Hereinafter, for convenience of description, it is assumed that acurrent block has a 16×16 size. The current block is divided into four8×8 transform blocks (Tbs). A transform is performed per 8×8 block.Thus, a total of four transforms are performed. To be prepare fortransformation, a current block can be divided into multiple transformblocks smaller than the size of the current block. Accordingly, when anintra prediction mode is determined in units of current blocks, thedetermined intra prediction mode is applied in units of transformblocks, and the actual prediction is performed in units of transformblocks. This scheme has an advantage of compensating for the defect thatthe correlation between pixels is decreased when the distance betweenthe reference pixel and the current block is increased. Referring toFIG. 21 , when intra prediction is performed on a block-by-block basis,the pixels of the transform block A are closer to the reference pixelsthan the pixels of the transform block D. Since the pixels in thetransform block D are far from the reference pixels, the predictionefficiency for the pixels in the transform block D is lowered.

In order to compensate for the above-described drawback, only the intraprediction mode is determined on a block-by-block basis, and the intraprediction is performed on a per transform block basis.

FIG. 21 shows the case where at least one reference pixel line selectedfor intra prediction of a current block is used for all the transformblocks within the current block when intra prediction is performed on aper transform block basis.

FIGS. 22A to 22D show the case where at least one reference pixel lineis selected for each transform block and is used for intra prediction.

Referring to FIG. 22A, when four transform blocks, each having a size of8×8, are used and prediction is performed on a transform block basis,the same reference pixel lines as shown in FIG. 21 are used for intraprediction of the transform block A.

Referring to FIG. 22B, for the transform block B, intra prediction isperformed by using the pixels of the reconstructed transform block A andthe reference pixel lines as illustrated. Similarly, in FIG. 22C, intraprediction is performed by using the pixels of the reconstructedtransform blocks A and B and the reference pixels lines as illustrated.In FIG. 22D, intra prediction is performed by using the pixels of thereconstructed transform blocks A, B, and C and the reference pixel linesas illustrated.

As described above, in the case of using multiple reference pixel lines,the pixel lines determined on a block-by-block basis as illustrated inFIG. 21 can be used as they are in the case of performing prediction ona per transform block basis as illustrated in FIGS. 22A to 22D.Alternatively, it is also possible to obtain an optimum reference pixelline for each transform block. Alternatively, information on whether theoptimum reference pixel line selected for each block unit is used forall of the transform blocks or a new optimum reference pixel line isderived for each transform block is encoded into a block header or anupper layer header for notification to the image decoding apparatus 600.

Next, various embodiments and application examples related to thederivation, encoding, and decoding of the intra prediction modeaccording to the present invention will be described with reference tothe drawings. When the intra prediction mode derivation method accordingto the present invention is used, the image encoding apparatus and theimage decoding apparatus can derive an intra prediction mode by usingthe same method and/or on the same criterion. Therefore, it is notnecessary to transmit information required for notification of an intraprediction mode to the image decoding apparatus.

<Derivation of Intra Prediction Mode by Image Decoding Apparatus>

In the present embodiment, a method in which an intra prediction mode isderived by an image decoding apparatus is described. The derivation ofan intra prediction mode by the image decoding apparatus, according tothe present invention, is referred as decoder-side intra mode derivation(DIMD). However, the DIMD can also be performed by the image encodingapparatus. Accordingly, the DIMD can be performed by the image encodingapparatus 100 as well as the image decoding apparatus 600, despite thename of DIMD. In particular, the DIMD may be performed by the intraprediction unit 102 of the image encoding apparatus 100 and the intraprediction unit 607 of the image decoding apparatus 600 in the samemanner.

Hereinafter, various embodiments of the DIMD according to the presentinvention will be described with reference to the drawings.

First Embodiment

FIG. 23 is a diagram illustrating DIMD according to a first embodimentof the present invention. According to an embodiment of the presentinvention, an intra prediction mode of a current block can be derivedusing already reconstructed pixels located around the current block.

Referring to FIG. 23 , the size of a current block 2001 to be encoded ordecoded is assumed to be M x N, the size of a template A 2004 is assumedto be P x N, and the size of a template B 2003 is assumed to be M x Q.As illustrated in FIG. 23 , a reference pixel region 2002 is composed ofa region located on the left side of the template A 2004 and a regionlocated above the template B 2003.

Assuming that the values of R and S that represent the size of thereference pixel region 2001 are respectively 1 and 1, the referencepixel region 2002 includes as many reference pixels as 2(Q+N)+2(P+M)+1.

According to an embodiment of the present invention, predicted values ofpixels within the template A 2004 and the template B 2003 are calculatedusing the reference pixels within the reference pixel region 2002according to each of the available intra prediction modes. In this case,the template A 2004 and the template B 2003 are treated as one region.For example, predicted values of the pixels within the template A 2004and the template B 2003 are calculated using the reference pixels withinthe reference pixel region 2002 according to each of 35 intra predictionmodes illustrated in FIG. 2 . For each mode of the 35 intra predictionmodes, the sum of absolute difference (SAD) which is the sum of thedifference between the original value and the reconstructed value of thetemplate A 2004 and the difference between the original value and thereconstructed value of the template B 2003 is calculated. Then, theintra prediction mode having the least SAD is selected as the intraprediction mode of the current block 2001.

Since the intra prediction mode of the current block 2001 is derivedusing the pixels reconstructed by the image coding apparatus 100 or theimage decoding apparatus 600, both of the image coding apparatus 100 andthe image decoding apparatus 600 can derive the same intra predictionmode.

On the other hand, the intra prediction mode used for luminance pixelscan be used for chrominance pixels. Since the intra prediction mode isnot transmitted to the image decoding apparatus 600 from the imageencoding apparatus 100, there is no overhead burden. Thus, it is alsopossible to add an intra prediction mode at a ½ position, ⅓ position, or¼ position between adjacent angular modes. Information on the number ofintra prediction modes used at this time can be transmitted to the imagedecoding apparatus 600 in various ways. For example, the information isencoded into the header of a block or an upper layer block, such as aslice header, a picture header, or a sequence header, using the repeatedmultiplication of 2 (i.e., 2 to the n-th power, 21. The encodedinformation is transmitted to the image decoding apparatus 600.Alternatively, information on the number of available intra predictionmodes can be transmitted to the image decoding apparatus 600 in a mannerof transmitting an index indicating one of multiple intra predictionmode lists composed of different numbers of intra prediction modes.

In the above-described embodiment, two templates including the templateA 2004 and the template B 2003 are used to derive the intra predictionmode of the current block 2001. However, three or more templates may beused.

In the above-described embodiment, the final intra prediction mode isderived by applying all of the intra prediction modes to the template A2004 and/or the template B 2003. However, the final prediction mode ofthe current block 2001 may be derived not by using all the intraprediction modes but by using only some selected intra prediction modes.

Second Embodiment

In the first embodiment described above, the two templates (template A2004 and template B 2003) are regarded as one region. However, in asecond embodiment, two templates (template A 2004 and template B 2003)are treated as separate regions. Specifically, two prediction modes arederived for a current block by using the two templates, respectively,and one of the two prediction modes is selected as the final predictionmode.

For example, referring to FIG. 2 , only the angular modes on the leftside of Mode 18 are used to obtain SADs for the template A 2004. Amongthe values of the SADs calculated for the respective angular modes onthe left side of Mode 18, a mode with which the least SDA value iscalculated is determined as the intra prediction mode of the template A2004.

Next, only the angular modes on the right side of Mode 18 are used toobtain SADs for the template B 2003. Among the values of the SADscalculated for the respective modes on the right side of Mode 18, a modewith which the least SDA value is calculated is determined as the intraprediction mode of the template B 2003. Next, one of the intraprediction mode of the template A 2004 and the intra prediction mode ofthe template B 2003 is finally selected as the prediction mode of thecurrent block 2001.

The SAD value corresponding to the DC mode and the SAD valuecorresponding to the planar mode are obtained by applying the DC modeand the planar mode to each template. Next, the SAD value correspondingto the mode selected as the intra prediction mode of a corresponding oneof the templates among the angular modes, the SAD value corresponding tothe DC mode, and the SAD value corresponding to the planar mode arecompared with each other. Thus, the final intra prediction mode of eachtemplate is selected.

Third Embodiment

Hereinafter, a third embodiment regarding DIMD, according to the presentinvention, will be described.

The third embodiment according to the present invention relates to amethod of executing DIMD using only the available templates when a partof the templates for a current block are not available.

FIG. 24 is a diagram illustrating DIMD according to a third embodimentof the present invention.

Referring to FIG. 24 , the template on the upper side of the currentblock 2101 is not usable, and only the template A 2004 on the left sideis available.

When the image encoding apparatus 100 and the image decoding apparatus600 use 35 intra prediction modes as illustrated in FIG. 2 and any oneof the two templates is unavailable, the 35 intra prediction modes areapplied to only one template 2104 by setting a range as shown in FIG. 24. In FIG. 24 , the left lower 45° direction is set as intra predictionmode 2, the horizontal direction is set as intra prediction mode 34, and33 angular modes exist between mode 2 and mode 34.

Ad described above, DIMD is performed by applying the 33 angular modes,the DC mode, and the planar mode to the template A 2104 to derive anintra prediction mode of the current block 2101.

When the current block 2101 corresponds to the upper boundary of aninput image, reference pixels 2105 on the left side of the template A2104 are available but reference pixels on the upper side of thetemplate A 2104 are not present. In this case, upper reference pixels2102 can be generated by padding appropriate neighboring pixels. Theneighboring pixels used for the padding may be pixels positioned on theupper side of the current block 2101 and/or the template A 2104, orreference pixels 2105 positioned on the left of the template A 2104.

Fourth Embodiment

FIG. 25 is a flowchart illustrating DIMD according to the presentinvention. This embodiment is related with the first through thirdembodiments described above. The method illustrated in FIG. 25 can beperformed in the same manner by the intra prediction unit 102 of theimage encoding apparatus 100 and the intra prediction unit 607 of theimage decoding apparatus 600.

Referring to FIG. 25 , an intra prediction mode of a reconstructed pixelregion is derived on the basis of a reference pixel region of at leastone reconstructed pixel region (S2201). Here, at least one pixel regionmay be the template 2004 or 2104 and/or the template B 2003 described inthe previous embodiments. However, the present invention is not limitedto these templates. The reference pixel region may correspond to thereference pixel region 2002, 2102, or 2105 described in the previousembodiments. However, the present invention is not limited to thesereference pixel regions.

Next, an intra prediction mode of the current block is derived on thebasis of the intra prediction mode of the reconstructed pixel region,which is derived in Step S2201 (S2203). After obtaining the intraprediction block of the current block using the derived intra predictionmode (S2205), the current block is reconstructed by summing the obtainedintra prediction block and a residual block of the current block(S2207).

Fifth Embodiment

FIG. 26 is a flowchart illustrating a method of encoding an intraprediction mode when an image is encoded using the DIMD according to thepresent invention.

First, information indicating whether to perform the DIMD according tothe present invention is encoded (S2501). This information is used toinform the image decoding apparatus 600 of an intra prediction modederivation method. That is, whether the intra prediction mode is derivedusing the DIMD according to the present invention or another method issignaled.

After determining whether the DIMD according to the present invention isused (S2502), when it is determined that the DIMD has been used, theprocess ends and the intra prediction mode of the current block isderived through the DIMD.

However, when it is determined that the DIMD according to the presentinvention has not been used, information on whether or not MPM (MostProbable Mode) is applied is encoded (S2503). As an alternative to theDIMD-based intra prediction mode derivation method, MPM can be used.

An MPM flag and MPM index information indicating whether the intraprediction mode of the current block belongs to a most probable mode(MPM) list are transmitted to the image decoding apparatus 600. Thenumber of intra prediction modes included in the MPM list is quite smallcompared to the total number of intra prediction modes. Therefore, whenthe intra prediction mode of the current block belongs to the MPM list,it is possible to signal to the image decoding apparatus 600 using muchfewer bits. MPM index information represents that the intra predictionmode of the current block corresponds to which mode among the modesbelonging to the MPM list.

When the MPM flag is 1, the intra prediction mode of the current blockbelongs to the MPM list. When the flag is 0, the intra prediction modeof the current block belongs to a group of the residual modes. The groupof the residual modes includes all intra prediction modes other than theintra prediction modes belonging to the MPM list. In Step S2503,encoding of the most probable mode (MPM) is performed by encoding theMPM flag.

Referring to FIG. 26 , after checking whether or not the MPM is used(S2504), when not used, the remaining modes other than the MPMcandidates are realigned and the intra prediction mode of the currentblock is encoded (S2505). When the MPM is used, the MPM index indicatingwhich intra prediction mode candidate has been used is encoded (S2506)and the process ends.

Sixth Embodiment

FIG. 27 is a flowchart illustrating a method of decoding an intraprediction mode when an image is decoded using the DIMD according to thepresent invention.

First, information indicating whether to perform the DIMD according tothe present invention is decoded (S2601). This information indicateswhether the intra prediction mode is derived using the DIMD according tothe present invention or another method.

After determining whether the DIMD according to the present inventionhas been used (S2602), when it is determined that the DIMD has beenused, the process ends and the intra prediction mode of the currentblock is derived through the DIMD.

However, when it is determined that the DIMD according to the presentinvention has not been used, information on whether or not the mostprobable mode (MPM) is used is decoded (S2603). In Step S2603, an MPMflag is decoded.

When the MPM flag is 1, the intra prediction mode of the current blockbelongs to the MPM list. When the flag is 0, the intra prediction modeof the current block belongs to a group of the residual modes. The groupof the residual modes includes all intra prediction modes other than theintra prediction modes belonging to the MPM list.

Next, it is checked whether or not the MPM has been used (S2604). Whenit is determined that the MPM has not been used, the remaining modesother than the MPM candidates are re-aligned and the intra predictionmode of the current block is decoded (S2605). When it is determined thatthe MPM has been used, the MPM index indicating which intra predictionmode candidate has been used is decoded (S2606) and the process ends.

Seventh Embodiment

FIG. 28 is a diagram illustrating a seventh embodiment of the presentinvention. The seventh embodiment of the present invention relates to amethod of deriving an intra prediction mode using multiple referencepixel lines and a template.

As shown in FIG. 28 , it is assumed that two reference sample linesincluding reference pixel line 1 and reference pixel line 2 are used asthe reference pixel lines of the template. A case where the values of Pand Q which represent the same of the template are all 1 will bedescribed.

Angular modes existing on the right side of the mode of the upper left45° direction are defined as upper-side angular modes. When theseangular modes are used, a template B and upper-side reference pixellines are used. Angular modes existing on the right side of the mode ofthe upper left 45° direction are defined as left-side angular modes.When these angular modes are used, a template A and left-side referencepixel lines are used. Next, prediction is performed for each referencepixel line as shown in FIG. 28 according to the intra prediction modes.For example, the reference pixel line 1 of the template is predictedusing the reference pixel line 2 of the template, and an optimal intraprediction mode candidate 1 is generated. Next, the template ispredicted using the reference pixel line 1 of the template, and anoptimal intra prediction mode candidate 2 is generated. When the intraprediction mode candidates 1 and 2 are the same, this prediction mode isselected as the intra prediction mode of the current block. When theintra prediction mode candidates 1 and 2 are not the same, it isdetermined whether to perform one of the DIMD methods according tovarious embodiments of the present invention described above.

FIG. 29 is a flowchart illustrating a process of encoding an intraprediction mode when the seventh embodiment is used.

First, intraframe prediction mode candidates are derived using areference pixel line of a template (S2801). Next, it is determinedwhether intra prediction mode candidates are the same (S2802). When theyare the same, the same mode is selected as the intra prediction mode ofthe current block. When the intra prediction mode candidates are not thesame, information on whether to perform the DIMD according to thepresent invention is encoded (S2803).

Since the subsequent steps S2804 through S2808 are substantially thesame as S2502 through S2506 illustrated in FIG. 26 , detaileddescription thereof will be omitted.

FIG. 30 is a flowchart illustrating a process of decoding an intraprediction mode when the seventh embodiment is used. The steps in theflowchart of FIG. 30 are substantially the same as the steps in theflowchart of FIG. 29 , respectively, except that the steps in theflowchart of FIG. 30 are performed by the image decoding apparatus 600rather than the image encoding apparatus. Therefore, detaileddescription of each step in FIG. 30 will be omitted.

<Modification to DIMD: Transmission of Template Index>

When one of the DIMDs according to the above-described variousembodiments is used, the intra prediction mode itself is not signaled.However, in the present embodiment, intra prediction mode candidates arederived using multiple templates, and index information indicating whichof the candidates is used is transmitted to the image decoding apparatus600 from the image encoding apparatus 100. FIGS. 31A and 31B arediagrams illustrating a modification to the DIMD, in which a templateindex is transmitted. Referring to FIG. 31A and FIG. 31B, intraprediction mode candidates are generated using two templates and theDIMD according to the present invention, and an index is used toindicate the candidate to be used as the intra prediction mode of thecurrent block. It is assumed that R and S representing the size of areference pixel region of the template shown in FIG. 31A or 31B are both1 for convenience of description.

In FIG. 31A, an intra prediction mode candidate 1 of a template A 3102is derived by using the pixels within a template reference pixel region3103. In FIG. 31A, an intra prediction mode candidate 2 of a template B3104 is derived by using the pixels within a template reference pixelregion 3105. Next, an index indicating that, among the intra predictionmode candidates derived from the multiple templates, the intraprediction mode derived from which template is used as the intraprediction mode of the current block 3101 is encoded, and the encodedindex is transmitted to the image decoding apparatus 600.

On the other hand, information on with which intra prediction mode ablock including the template is encoded can be used in this embodiment.For example, in FIG. 31A and FIG. 31B, assuming that an intra predictionmode of a block including the template A 3102 is an intra predictionmode A, when the intra prediction mode A is the same as the intraprediction mode that is derived by applying the template referencepixels to the template A 3102, a higher or lower priority can be given.It is possible to allocate bits at the time of arranging candidates forindex setting according to the priority. Similarly, when there aremultiple templates including the template A 3102, it is possible to setpriorities at the time of allocating bits using the above-describedcriterion.

FIG. 32 is a flowchart illustrating a method of encoding an intraprediction mode according to a DIMD in which a template index is used.

First, information indicating whether or not the DIMD in which thetemplate index is used is encoded (S3301). After determining whether theDIMD in which the template index is used has been performed (S3302),when it is determined that the DIMD has been performed, the templateindex is encoded and the process ends (S3307). However, when it isdetermined that the DIMD has not been performed, information on whetheror not to apply the most probable mode (MPM) is encoded (S3303). Next,it is checked whether the MPM has been applied (S3304). When it isdetermined that the MPM has not be applied, the remaining modes exceptfor MPM candidates are re-arranged and encoded (S3305). In contrast,when it is determined that the MPM has been applied, an MPM indexindicating which candidate is applied is encoded (S3306), and theprocess ends.

FIG. 33 is a flowchart illustrating a method of decoding an intraprediction mode according to a DIMD in which a template index is used.The steps in the flowchart of FIG. 33 are substantially the same as thesteps in the flowchart of FIG. 32 , respectively, except that the stepsin the flowchart of FIG. 33 are performed by the image decodingapparatus 600 rather than the image encoding apparatus. Therefore,detailed description of each step in FIG. 33 will be omitted.

Although FIGS. 31A and 31B illustrate that two templates adjacent to thecurrent block are used, three or more templates may be used.

<Application Example of DIMD: Generation of MPM List>

In the methods described above, when the DIMD according to the presentinvention is used, the intra prediction mode itself is not signaled tothe image decoding apparatus 600, or template index informationindicating the intra prediction mode candidate derived by using whichtemplate of multiple templates is selected as the intra prediction modeof the current block is transmitted to the image decoding apparatus 600from the image encoding apparatus 100.

Hereinafter, an embodiment will be described in which most probable modecandidates are re-arranged or an MPM list is generated using intraprediction modes derived according to the DIMD.

It is also possible to arrange the MPM candidates in the order in whichthe intra prediction modes derived by using the templates are arrangedafter generating the MPM candidates for prediction of the intraprediction mode.

First Embodiment

FIG. 34 is a diagram illustrating an example of a method of setting anintra prediction mode derived by using a template as an MPM candidate.

It is assumed that a reconstructed block A 3503 and a reconstructedblock B 3505 are present around a current block 3501, the block A 3503is predicted through intra prediction, and the block B 3505 is predictedthrough inter prediction. Since the block B 3505 is predicted throughinter prediction, it does not have an intra prediction mode. Therefore,in the present embodiment, the intra prediction mode of the block B 3505is generated using the template. The method of deriving the intraprediction mode of the block B 3505 by using the template can beunderstood by referring to the above-described methods. On the otherhand, when deriving the intra prediction mode of the block B 3505 byusing the template B 3104 of FIG. 31B, the intra prediction modeencoding method, the number of intra prediction modes, and theprediction angles need to be the same as those preset by the imageencoding apparatus 100 or the image decoding apparatus 600.

Second Embodiment

In this embodiment, MPM candidates are determined depending on withwhich inter prediction mode a block containing a template is encoded.

FIG. 35 is a diagram illustrating a method of setting an MPM candidateaccording to an embodiment of the present invention. Referring to FIG.35 , the intra prediction mode of the left-side block 3703 of thecurrent block 3701 is assumed to be Mode 10, and the intra predictionmode derived by using template reference pixels (not illustrated) withina template A 3705 which is a portion of the left-side block 3703 isassumed to be Mode 12. Similarly, the intra prediction mode of theupper-side block 3707 of the current block 3701 is assumed to be Mode28, and the intra prediction mode derived by using template referencepixels (not illustrated) within a template B 3709 which is a portion ofthe upper-side block 3707 is assumed to be Mode 28. In the case of theleft-side block 3703, considering the entirety of the left-side block3703, Mode 10 is advantageous for encoding over Mode 12. However, sinceMode 12 is set for the template A 3705 adjacent to the current block3701, it can be considered that Mode 12 is more appropriate as theprediction mode of the current block than Mode 10. In this case, the MPMcandidate is generated using the intra prediction mode derived from thetemplate A 3705 rather than the intra prediction mode of the left-sideblock 3703 when the MPM candidates are set.

In the case of the upper-side block 3707, since the intra predictionmode of the upper-side block 3707 is the same as the intra predictionmode derived from the template B 3709, the same intra prediction mode isused as the MPM candidate.

As an alternative, the MPM candidates can be set by using four modesincluding the intra prediction mode of the left-side block 3703, theintra prediction mode derived from the template A 3705, the intraprediction mode of the upper-side block 3707, and the intra predictionmode derived from the template B 3709. In this case, since the intraprediction mode derived by using the template is closer to the currentblock, the intra prediction mode derived by using the template is givena higher priority and a lesser number of bits is allocated for thederived intra prediction mode at the time of setting the MPM candidates.

Although the exemplary methods of the present disclosure are representedby a series of steps for clarity of description, they are not intendedto limit the order in which the steps are performed. That is, ifnecessary, each step may be performed in parallel or performed in seriesin a different order. In order to implement the method according to thepresent disclosure, each of the embodiments described above can bemodified such that some additional steps can be added to a correspondingembodiment or some existing steps can be eliminated from a correspondingembodiment. Alternatively, some additional steps are added and someexisting steps are eliminated from a corresponding of the embodiments.

Various embodiments in the present disclosure are not intended torepresent all of the possible combinations based on technical spirit ofthe present invention but are provided only for illustrative purposes.Elements or steps described in various embodiments can be appliedindependently or in combination.

Various embodiments in the present disclosure can be implemented byhardware, firmware, software, or a combination thereof. When implementedby hardware, each of the embodiments can be implemented by one or moreapplication specific integrated circuits (ASICs), digital signalprocessors (DSPs), digital signal processing devices (DSPDs),programmable logic devices (PLDs), field programmable gate arrays(FPGAs), general processors, controllers, micro controllers, ormicro-processors.

The scope of the present disclosure covers software ormachine-executable commands (for example, operating systems (OSs),application programs, firmware, programs) that enable steps in variousembodiments to be performed in a certain device or computer, and anon-transitory computer-readable medium in which such software orcommands are stored so as to be executable in a certain device orcomputer when read out.

INDUSTRIAL APPLICABILITY

The present invention can be used for video signal encoding or decoding.

1. A video decoding method performed by a video decoding apparatus, thevideo decoding method comprising: selecting at least one reference pixelline among a plurality of reference pixel lines, the plurality ofreference pixel lines being located in the same image as a current blockto be decoded using intra prediction; deriving a predicted value of atleast one pixel in the current block based on a value of at least onepixel included in the selected at least one reference pixel line; andreconstructing the current block based on the predicted value, whereinthe deriving the predicted value is performed in units of subblocks inthe current block.
 2. The method of claim 1, further comprising:obtaining reference pixel line index information from an inputbitstream, wherein the at least one reference pixel line is selectedamong the plurality of reference pixel lines based on the obtainedreference pixel line index information.
 3. The method of claim 1,wherein the plurality of reference pixel lines include reference pixellines at predetermined positions.
 4. The method of claim 1, wherein anumber of the plurality of reference pixel lines is
 3. 5. The method ofclaim 2, wherein the reference pixel line index information for each ofthe plurality of reference pixel lines has a value proportional to adistance between the each of the plurality of reference pixel lines andthe current block.
 6. A video encoding method performed by a videoencoding apparatus, the video encoding method comprising: selecting atleast one reference pixel line among a plurality of reference pixellines, the plurality of reference pixel lines being located in the sameimage as a current block to be encoded using intra prediction; derivinga predicted value of at least one pixel in the current block based on avalue of at least one pixel included in the selected at least onereference pixel line; and encoding the current block based on thepredicted value, wherein the deriving the predicted value is performedin units of subblocks in the current block.
 7. The method of claim 6,further comprising: encoding reference pixel line index informationspecifying the selected at least one reference pixel line.
 8. The methodof claim 6, wherein the plurality of reference pixel lines includereference pixel lines at predetermined positions.
 9. The method of claim6, wherein a number of the plurality of reference pixel lines is
 3. 10.The method of claim 7, wherein the reference pixel line indexinformation for each of the plurality of reference pixel lines has avalue proportional to a distance between the each of the plurality ofreference pixel lines and the current block.
 11. A non-transitorycomputer-readable recording medium storing a bitstream generated by avideo encoding method, the video encoding method comprising: selectingat least one reference pixel line among a plurality of reference pixellines, the plurality of reference pixel lines being located in the sameimage as a current block to be encoded using intra prediction; derivinga predicted value of at least one pixel in the current block based on avalue of at least one pixel included in the selected at least onereference pixel line; and encoding the current block based on thepredicted value, wherein the deriving the predicted value is performedin units of subblocks in the current block.